English/Reading/Journalism Assessment

With exactly 20 years of history behind its assessment activities, GCC’s English Department has always been a leader in the field of assessment. The department mandates that each instructor must participate in assessment activities each semester for each course taught. During the past several academic years, instructors teaching core courses including ENG 101 and ENG 102 were expected to do two things in order to fulfil the department’s assessment initiatives; instructors had to attend the Assessment Day activities during the fall and spring sessions, and they were also expected to complete group or individual assessment activities. Instructors teaching other ENH, ESL, Dev.Ed., RDG, CRE, and JRN courses were expected to attend the assessment day sessions and also complete individual assessment forms. However, for the past several semesters, participation in assessment day activities and individual or group assessment options has seen a steady decline. Poor timing, procedural difficulties, limited buy-in, and lack of accountability were cited as important factors contributing to this decline in participation. In order to boost participation and gear up for the impending Higher Learning Commission (HLC) visit, the English Assessment Committee introduced several new assessment initiatives during the year 2008-2009.

A report of department-wide participation in assessment activities for the year 2008-2009 follows:

During the Fall 2008 Assessment Day session, 33 full-time, 86 part-time, and 4 active retiree faculty were on the department payroll. However, only 62 faculty members participated, approximately 50.40 %. Though some individuals were either obligated to participate in other college activities or did not receive a contract prior to the event, low numbers prompt a re-evaluation of our approach.

As expected, the common grading activity also sparked off a resounding debate and discussion about establishing common rubrics and department-wide grading guidelines. In the past, rubrics established by the committee have been provided for these large group grading sessions. In these instances, the bulk of the dialogue during open discussion has been about how to interpret the rubric and how to establish shared assumptions about instructor expectations and the conditions present while the essay was written. Faculty members have consistently struggled with interpretations of criteria offered in a shared rubric. For example, instructors have distinctly different views on what “inadequate” constitutes in the following phrase from the rubric—“Diction is inadequate.” Equally challenging has been addressing the diversity of approaches to the English 101 Common Assignment. Though the topic, in the past, has been pre-determined for participants and outside sources have been provided, the specific requirements/guidelines have been largely shaped by the individual instructor. For instance, some instructors required five outside sources for the argumentation while others required two. Time allotted, process entailed, feedback provided, and rhetorical patterns prescribed varied significantly between English 101 instructors. Even the most basic assumptions became problematic--differences in minimum word count, number of paragraphs required, placement of thesis statement, the use of first-person, and the level of mastery of Competency Four (in particular the skills listed under IV). Therefore, in Fall 2008, the Assessment Committee believed it necessary to approach the large grading session differently.

Faculty members attending the large grading session “Deal or No Deal: Assessing Our Grading Practices” were given four essays pre-selected by the committee from the ENG 101 Common Final Argumentation submissions (Spring ’08). Strict anonymity with reference to student/instructor names were maintained during this activity. Participants were asked to grade the essays individually and then to discuss their feedback in groups of five or six. During both the individual and the group discussion, they were to arrive at a P/F. Following the group activity component, individual groups were asked to disseminate for the whole group how they determined the grade.

The task was reduced to arriving at a P/F to ensure that the focus remained on what faculty considered a competent piece of writing from an ENG 101 student toward the end of the semester. The assumptions generated by the committee provided context for the instructors; thus, discussion no longer gravitated toward elaborating on differences in expectations and requirements.

Thirty-four faculty members participated. Three of the four essays were discussed at length following the individual grading. Paper #4 could not be read due to lack of time. Each group presented their justification for the grade. There was 100% agreement on three of the essays. All six groups failed the papers, citing similar reasons. The majority (89%) of the reasons presented concerned the following competencies:

Part II of the grading session involved individual grading of an essay pre-selected by the committee members from the Spring ’09 common final submissions. Again, strict anonymity with reference to student/instructor identity was maintained. Participants were encouraged to respond to the essay as they typically do when evaluating and commenting on work submitted by one of their students, with an understanding that the student would have additional writing assignments to fulfill that semester. Thus, the instructor’s communication would reflect teaching strategies normally implemented when working with their own students.

Participants were told that if they typically write comments, mark sentence-level errors, and/or provide annotations or end-notes, they should do so during this activity. At the same time, efforts were made to not influence their responses. For example, if an instructor doesn’t typically identify sentence-level errors for their students and instead writes in the end-note the types of errors made, he/she was encouraged not to change their grading practices/philosophies. If the faculty member usually did not generate an end note for the student with “justification” for the grade, the committee asked that he/she complete an evaluation form for us providing details/concerns justifying/explaining.

Out of the 34 participants 33 failed the paper. One participant did not provide an F/P grade. More than likely this was an oversight.

First, significant agreement among faculty was evident. The most obvious example of agreement was that 100% of the instructors who provided a P/F grade failed the essay. Twenty-six out of thirty-four (77%) wrote margin comments to the student indicating where he/she digressed from their topic. Phrases used by the instructors were similar: “Focus?;” “Not on subject;” “Off topic;” “This paragraph does not relate to the topic;” and “What is the significance of this paragraph to the paper?”
Though only 18 out of 34 (53%) of the participants provided margin comments concerning the thesis statement, more than 90% discussed it in the “Justification” and “Weaknesses” sections of the evaluation form provided by the committee. Again, margin comments reflected agreement in grading practices among faculty—“Thesis?;” “Is this your thesis? Unclear;” “What is the thesis?;” and “Thesis statement is too broad—be more specific.”
When presenting the justification for an “F” rating, 29 out of 34 (85%) emphasized lack of focus. They indicated the thesis statement did not clarify stance, nor did the body of the essay stay on topic (“lack of coherence;” unorganized,” “unrelated content,” were phrases included).
Only 5 out of 34 (15%) faculty members stated that the essay was not competent because of grammatical and punctuation errors

Only nine out of 34 (26%) faculty members provided end comments for the students. Individuals who offered an end comment also were more likely to offer in-depth margin comments.
In contrast, 21% of the participants offered no margin comments and did not mark (underline, circle, or offer a symbol indicating a sentence-level error) within the text beyond the first page. When examining the comments (margin, end, textual, and evaluation form), an additional trend emerged.
Only five out of 34 faculty members offered at least one reinforcing comment, reinforcing referring to skills demonstrated. All other comments were redirecting, comments focused on skills not demonstrated and directive in how to improve the writing.

Participants of the ENG 102 large group break-out session with pre-selected work written by Spring 2008 students. The student work (numbered 1-4) was comprised of summaries of the article, “For Teenagers, the Car is the Danger Zone” from the pool of work submitted by instructors participating in the ENG 102 Common Assessment Activity. Anonymity for both the student and the instructor was maintained.

Participants were asked to read the article and then consider the student summaries carefully, noting their strengths and weaknesses on an evaluation form provided to them by the committee. Following assessing the work individually, they discussed and scored the summaries P/F as a group.

Each group chose one summary to discuss with the large group. During that time, faculty members were asked to debrief the larger group on identified strengths and weaknesses of the summary and offer justification for the grade.

Though no rubric was provided, traits of a competent summary were provided. Participants were to rely on their individual grading rubric/practices and interpretations of the traits described above.

There was significant agreement among the 28 faculty members who participated in the session. When evaluating Summary #1-Summary #3. In general, the agreement was expressed in two ways—the strengths and weaknesses recognized and the grade (P/F).

For Summary #1, 89% of the instructors reporting back determined it was competent, thus a “P.” Strengths listed by both groups (those passing and those failing the writing) were similar. They recognized the voice (language choice) and the grasp of the main point as positives. In addition, the weaknesses highlighted were the same—the summary lacked supporting detail/data.
Summary #2, conversely, was failed by 88% of the evaluators. Both the instructors failing and passing the summary recognized the weaknesses to be no citation and grammar/punctuation errors. They did not agree, however, on strengths. Individuals passing #2 argued that the main points were accurate and clear, while those who failed the summary asserted the student missed the main point.
Greatest agreement emerged when discussing Summary #3; 100% of the instructors failed it because the student failed to identify and express the main point, to demonstrate command of grammar and punctuation conventions, and to provide a citation.
Evaluation of Summary #4 resulted in a split; 50% failed and 50% passed the summary. Strengths and weaknesses reported differed significantly between the two groups. Those passing the summary listed the command of the main ideas, organization, knowledge of the author and source, and the controlling ideas as strengths. Instructors failing it emphasized these strengths: command of format and documentation, effective use of transitions, and a compelling writing style. Less contrasting were the reported weaknesses; both listed the flooding of details as distracting and disconnected from the larger focus. The other weakness mentioned by individuals giving the summary a failing grade was that paraphrased material too closely matched the original text.

Faculty members were asked to read Summary #5 carefully and grade it individually. They were encouraged to evaluate and comment on the summary in a manner that was typical of their grading practices. If they usually made corrections, wrote margin comments, and/or end-notes, they were to do so on the paper as if offering feedback to a student in their class. Likewise, if their usual grading practices did not include comments on the paper, they were asked to justify the grade on the evaluation form. The assumptions (see previous section) presented during the large-group activity were also to be considered while completing the individual grading activity. As with the previous summaries, a determination of Pass or Fail was required.

Following evaluating the summary, participating instructors were asked to submit to the Assessment Committee the paper (with or without comments/markings, depending on typical grading practices) and the evaluation sheet.
As with the large-group activity, there was significant agreement.

93% of the participants passed the summary.
Individuals failing the summary cited the strengths as being thorough explanation and clear attempts to reveal original content.
The weaknesses discussed were that it was poorly written (grammar and punctuation errors), had weak transitions, and lacked sentence variety.
All in all, those who passed the summary agreed with the strengths and weaknesses listed by those failing the writing. They, too, recognized the writer’s attempt to use original language in order to capture the content of the text.
Organization and clear efforts to use MLA format correctly were other strengths discussed.
In addition to agreeing that there were errors in grammar, mechanics, and diction, they reported that there was no concluding sentence and lacked some detail.

Note: During the Assessment Day session, the RDG/CRE faculty and the Dev.Ed. faculty members worked on creating tools and guidelines for a common assignment in their areas of expertise.

For Fall 2008, the common ENG 101/107 assignment (group assessment option) was a documented argument essay. Unlike in previous years, there was no mandatory topic for the common assignment. The English Assessment Committee arrived at this decision based on comments/suggestions provided by part-time and full-time instructors during the previous Assessment Day feedback session. For the past several semesters, the committee members noted a downward slide in participation during common final activities. The majority of ENG 101 faculty members cited three primary reasons why they did not participate in the common assignment—1) the materials and topic were available too late in the semester to effectively integrate them into the course scheduling; 2) the topic either did not appeal to them or to the students; and/or 3) the method to submit the essays was belabored or confusing. The committee hoped that by giving ENG 101/107 instructors the freedom to provide a topic (or a list of topics) for their students, participation numbers would rise. Some instructors had also mentioned that they were opting out of the common final option because they liked to let their students pick relevant topics for the argument essays. The committee hoped that with no mandatory topic being prescribed, these instructors would also join the common final bandwagon. Guidelines and criteria were, however, provided by the committee. These were designed to establish minimums and a standardization that would make it possible to evaluate the argument essays with shared assumptions.
Other details about submission criteria for this assignment can be found online at the following web address:
http://web.gccaz.edu/English/Assessment/Fall08/Eng101.html
Instructors who chose the common final option had to report on their experiences after collecting their students’ final drafts by filling out the following form:
http://web.gccaz.edu/English/Assessment/Fall08/Reports/Fall08GroupRpt.html
Summarizing is a key skill in ENG 102 classes. It requires students to read carefully, to identify main ideas, and to paraphrase those ideas accurately. Therefore, the common assignment for English 102/108 for fall 2008 was writing summaries. The Assessment Committee had selected the sources for this activity. All the sources were available via GCC’s library databases. Details about submission criteria for this assignment can be found online at the following web address:
http://web.gccaz.edu/English/Assessment/Fall08/Eng102.html
ENG 102 instructors who participated in the common assignment had to report on their experiences after collecting their students’ final drafts by filling out the following form:
http://web.gccaz.edu/English/Assessment/Fall08/Reports/Fall08GroupRpt.html
At the end of the semester, the Assessment Committee members collected common final submissions from various ENG 101 and 102 sections. Samples were ear-marked for common grading activities during the Spring Assessment Day held during the week of accountability in January.
Instructors who were unwilling or unable to participate in the group/common ENG 101 and 102 assignments chose to submit individual assessment reports. Forms for individual submissions were made available online at the following web address:
http://www.gc.maricopa.edu/notebook/EnglishAssessment/
A synopsis of the participation record collated and compiled by the Assessment Committee members at the end of the Fall 2008 semester follows:

	Total # of faculty	Faculty filing individual reports [ENG, RDG, CRE, ESL and ENH classes]	Total participating
Full-Time Faculty	33 regular + 4 retirees = 37	16	43.24%
Part-Time Faculty	86	5	5.81%
Total	123	21	17.07%

Details about the individual assessment activities have been filed under the online Notebook
section on our web-page at the following web address: http://www.gc.maricopa.edu/notebook/EnglishAssessment/
Note: The numbers above represent the submissions the committee members received as of the first week of Jan ’09. We had a couple or so multiple submissions.

For Spring 2009, the assessment committee decided to pilot a new approach. One of the biggest ongoing debates during the grading sessions has been the lack of agreement in grading practices across the department. After reading through the feedback forms submitted by our colleagues during the previous several Assessment Day sessions, and from informal discussions with several faculty members, the committee concluded that most of the debates about grading practices stemmed from a lack of focus with reference to course competencies. For example, during the large group grading session in Fall 2008, though the pre-determined assumptions provided by the committee served to guide the discussion to the primary focus (assessing whether or not an essay is competent), the dialogue still revealed diverse interpretations of Competency 4:

4. Summarize, paraphrase and quote from sources to maintain academic integrity and to develop and support one’s own ideas. (III, IV)

Interpretation of “Knowing Conventions” and expectations concerning level of skill/sophisticated with which the student can “[s]ummarize, paraphrase and quote from sources to maintain academic integrity and to develop and support one’s own ideas” varied significantly.
With this in mind, the committee decided to shift gears and focus more on course competencies. The new approach also piloted a no common final/common assignment option for ENG 101/107 or ENG 102/108. Instead, instructors were asked to choose two course competencies to focus on at the beginning of the semester itself. They had to report on their choices within the first five weeks of the semester by using the following form.
http://web.gccaz.edu/English/Assessment/Spring09/Spr09IndvRpt.html
This form essentially showcases what the instructor proposed to do in his/her class with reference to course competency focus.
Instructors were also required to fill out a post-assessment form at the end of the semester in order to compare what they had planned to do at the beginning of the semester with what they actually did accomplish during the course of the semester with reference to course competency focus. Separate forms were required for each course. However, instructors were allowed to assess multiple sections of the same course using a single form. Details about the post-assessment form are available via the following url.
http://web.gccaz.edu/English/Assessment/Spring09/Spr09IndvRptPost.html
A synopsis of the participation record for the pre-assessment forms collated and compiled by the Assessment Committee members during Spring 2009 semester follows:

	Total # of faculty	Faculty filing pre-assessment reports [ENG, RDG, CRE, ESL and ENH classes]	Total participating
Full-Time Faculty	33 regular + 3 retirees = 36	18	50%
Part-Time Faculty	78	26	33.33%
Total	114	44	38.59%

A synopsis of the participation record for the post-assessment forms collated and compiled by the Assessment Committee members at the end of the Spring 2009 semester follows:

	Total # of faculty	Faculty filing pre-assessment reports [ENG, RDG, CRE, ESL and ENH classes]	Total participating
Full-Time Faculty	33 regular + 3 retirees = 36	14	38.88%
Part-Time Faculty	78	25	32.05%
Total	114	39	34.21%

Note: Participation numbers, while still well below the mandatory standards have shown some improvement. This could be attributed to the new initiatives adopted by the committee.

Spring Assessment Day: January 2009
34 faculty members (adjunct and residential combined) participated in the ENG 101 common grading activity.
From the materials submitted during Fall ‘08, the committee selected three graded essays (Essay #1— “B+/88%,” Essay #2—“C-/70%,” and Essay #3—“C/75%”—grades recorded on the submitted samples) from three different instructors for the assessment activity. The first stage involved having five veteran/tenured residential faculty members who consistently teach ENG 101 evaluate the three essays individually without knowing what grade had been assigned by the instructor. The second stage, taking place on Assessment Day during the ENG 101 break-out session, opened the evaluation of one of the essays to five groups of approximately five instructors. As groups, participants were asked to determine if the writing demonstrated competence (assigning a P/F rating) according to the MCCCD mandated ENG 101 exit competencies.

During stage one, all five veteran instructors passed Essay #1, four assigning a letter grade of “A” and one a “B.” With Essay #2, three failed and two passed the writing. Still, considering the original instructor assigned it a C-, the results reveal a great deal of agreement, the spread less than a letter grade difference. Concerning Essay #3, four participants passed the essay--one “B” and three “C’s.” One instructor failed the essay but said it was a strong “D.” Because of the varied responses (a two-letter grade range), Essay #3 was chosen for the ENG 101 break-out session.
As with past Assessment Day grading sessions, the large-group environment resulted in a harsher rating.

No group reported back that the essay was a solid “C,” and nearly 50% of the participants failed the writing.
There was disagreement within and between groups in terms of which ENG 101 exit competencies were not demonstrated.
In particular, nearly half of the participants stated the writing did not illustrate Competency 1, specifically that it failed to respond to the prescribed rhetorical context.
Most contentious was the discussion concerning Competency 4. Views differed significantly about what level of skill needed to be demonstrated to consider the writing competent in terms of summarizing, paraphrasing, and quoting “from sources to maintain academic integrity and to develop and support” the writer’s ideas. Not dissimilar to past Assessment Day discussions, instructors had diverse interpretations of 4 (1V.C.)—“Knowing Conventions” and “Documentation of sources.” Some stated they require the writing exhibit an attempt at MLA in-text citation and Works Cited format, but they do not expect command of the skill. Others asserted that ENG 101 was an opportunity to introduce the students to the conventions and the skills required to integrate outside sources effectively, but that it should not be the emphasis. Others, yet, expected a high level of understanding and mastery of the conventions.
However, all participants agreed that the writing exhibited aptitude in Exit Competencies 2 and 3. The writing was predominantly clean, focused, and clearly organized.

Once the groups reported the findings, the committee revealed both the original instructor’s and the veteran instructors’ ratings. This led to a productive discussion about the phenomenon of increased and the importance of arriving at shared interpretations of the competencies and level of aptitude of those skills needing to be evident in order for the writing to be considered competent (P) at an ENG 101 level.

The Spring 2009 Assessment Day ENG 102 break-out session focused on assessing ENG 102 instructors’ expectations of the writing skills of incoming students having successfully completed ENG 101. Approximately twenty faculty members participated. As discussed above, core to having consistent grading practices and expectations, it was essential to arrive at a shared interpretation of ENG 101 Exit Course Competencies and the required level of command of these prescribed skills to determine a piece of writing competent and a student ready to progress to 102.

Integral to determining ENG 102 instructors’ expectations was the need to assess their interpretation of Competency 4 listed in the ENG 101 Exit Course Competencies. Since discussions during ENG 101 large grading sessions consistently have gravitated to this important competency, it seemed essential to ascertain from the ENG 102 instructors their understanding of the competency and what level of knowledge/mastery they expected to be evident in the writings of incoming students.

The ENG 102 session was a data gathering endeavor. Participants were provided a copy of the MCCCD Official Course Competencies for ENG 101. By referring to the competencies below, ENG 102 instructors were to generate a list of the skills they expect incoming students to exhibit and deficiencies they consistently find.

Analyze specific rhetorical contexts, including circumstance, purpose, topic, audience, and writer, as well as the writing’s ethical, political, and cultural implications.
Organize writing to support a central idea through unity, coherence, and logical development appropriate to a specific writing context.
Use appropriate conventions in writing, including consistent voice, tone, diction, grammar, and mechanics.
Summarize, paraphrase, and quote from sources to maintain academic integrity and to develop and support one’s own ideas.
Use feedback obtained from peer review, instructor comments, and/or other methods to revise writing.
Assess one’s own writing strengths and identify strategies for improvement through instructor conference, portfolio review, written evaluation, and/or other methods.
Generate, format, and edit writing using appropriate technologies.

Though all seven ENG 101 competencies were covered, including the macro-perspective that an incoming student should be able to write an essay that exemplifies an understanding of all involved conventions (voice, thesis, organization, audience, and so on), particular skills were discussed at greater length.
Session participants emphasized that students entering ENG 102 should be able to: 1) identify and write for a particular audience (Competency 1), 2) apply all of the steps involved in the writing process (Competencies 1, 2, 3, 5, 6, 7, and often 4), 3) write a strong thesis statement (Competency 2), 4) offer coherent/developed paragraphs and essays (Competency 2), 5) implement an appropriate mode (Competencies 1 and 2), 6) read critically, including their own work (Competencies 1, 4, and 6), 7) avoid fragments, run-on sentences, comma splices, and double negation (Competency 3), and 8) use Blackboard (Competency 7).

The expectations listed above were also considered deficiencies evident in the writing of some incoming students. There were additional identified deficiencies. Participants of the break-out session asserted that student writing frequently failed to demonstrate one of the following traits: 1) logical development, 2) appropriate word choice/vocabulary, 3) academic integrity (plagiarism prevalent), 4) command of basic conventions, and 5) an understanding of audience in terms of academic versus informal (slang—“text message grammar”).

As instructed, the group discussed MCCCD ENG 101 official course Competency 4 in greater depth. Since many of the participants also teach ENG 101, they were prepared to examine expectations in two ways—what they cover in ENG 101 because of how they perceive ENG 101 competencies and what they expect a student to know when entering their ENG 102 classroom. The results were interesting. They expected less from incoming students than what they demanded of their exiting ENG 101 students. Many of them stated that their ENG 101 students were familiar and skilled at using outside sources when successfully completing the course.

However, for incoming students, the participants expected that they only have “some” sense of research skills, “some” skill at integrating and citing sources, and “some” sense of MLA (or APA if taught) and its conventions. They determined that ENG 101 competencies do not require students to be aware of MLA formatting or research in general. The competencies require the students to quote, paraphrase, and summarize, but not to research and document outside sources.

Six faculty teaching ENG071, one faculty teaching ESL077, four faculty teaching ENG061, and one faculty teaching ESL067 participated in the common final assessment activity. The ENG071/ESL077 common final was comparison/contrast essay. Faculty chose a topic from a list of “approved” topics and followed the same grading rubric, which mirrors the criteria required for an essay as presented in the common ENG071/ESL077 textbook. The ENG061/ESL067 common final was a paragraph based on a picture (the textbook uses pictures as a method of generating writing) of their choice. A common rubric was used to score the paragraphs and is based on the criteria required for a paragraph as presented in the common ENG061/ESL067 textbook. (Examples of the rubrics and writing prompts were supplied under separate cover.)

Common final submissions of students’ work (a “B,” “C,” and “D” paper) were collected from each of the participating faculty by David Miller, Co-ordinator of the Dev. Ed. Program at GCC.

In a continued effort to gather data concerning departmental trends so that greater agreement in approach and learning outcomes can be achieved, a comprehensive survey was generated and distributed to instructors who had taught ENG 101 in Spring 2009. The survey questions were designed to assess the methods and grading practices used by individual instructors when teaching the argument essay. Because the English Department expects all ENG 101 instructors to teach the argument essay and it has been the mode used most frequently as the 101 common assignment, data gleaned from the survey will be instrumental to understanding areas of agreement and disagreement in terms of approach and the interpretation of exit competencies.

Thirty-seven instructors taught ENG 101 during the Spring 2009 semester. Fourteen out of eighteen residential faculty (78%) and seventeen out of twenty-nine (59%) participated in this effort. The findings will be presented this fall, and a sub-committee will be formed to establish departmental expectations and grading practices so that the competencies are being taught and students successfully completing ENG 101 demonstrate command of the competencies.


	Last updated by Rashmi Menon on June 23, 2009. Legal Notice. http://web.gccaz.edu/English/Assessment/Spring09/Assessment Spring 2009.html Glendale Community College, Glendale, AZ